Proximal Gradient Methods with Adaptive Subspace Sampling

Many applications in machine learning or signal processing involve nonsmooth optimization problems. This nonsmoothness brings a low-dimensional structure to the optimal solutions. In this paper, we propose randomized proximal gradient method harnessing underlying structure. We introduce two key components: (i) random subspace algorithm; and (ii) an identification-based sampling of subspaces. Their interplay significant performance improvement on typical problems terms dimensions explored.